Field of the Invention
Embodiments of the invention described herein pertain to the field of computer systems. More particularly, but not by way of limitation, one or more embodiments of the invention enable video multi-encoders.
Description of the Related Art
Encoding and decoding are essential to representing video in digital form. Compression is a practical necessity for transmitting and storing digitized video efficiently and economically. This is true of both display-size extremes of today's video. Standard high definition video and emerging extra-high definition video require challenging trade-offs between compression (to satisfy storage or bandwidth limitations), decoder computational requirements, and viewing quality. At the opposite extreme, a proliferating mobile customer base competes for essentially fixed total bandwidth resources.
Many video codecs have arisen to address these needs, culminating in such codecs and codec standards as, MPEG-2, JPEG 2000 Video, VC-1, MPEG-4, H.264, and WebM. Video codecs are typically designed to offer new or improved features even when they are based on previous codecs or an assortment of ideas found in previous codecs. Nonetheless, each individual codec is intended to be complete in itself and to work independently of all other video codecs. This is also true of codecs satisfying codec standards, including JPEG 2000 Video, MPEG-2, MPEG-4, H.264, and the emerging H.265. As a result, a choice of codec typically involves a trade-off among desired features, for example, frame rate and display resolution versus content encryption, compression versus final video quality, real time video capture versus display and transmission bandwidth requirements, or any other feature.
There have been many successful efforts to develop products that involve multiple codecs, but the great majority addresses the problem of converting encoded video from one codec to another. In order to convert encoded video from codec Y to codec Z, the converter, or transcoder, behaves as if it were a two-stage serial device, the first stage of which is a Y-decoder, followed by a Z-encoder as the second stage. The features of the video product of such devices are usually typical of video products of the output encoder alone.
A few efforts have incorporated multiple codecs into a single system, allowing the system to behave like any one of them, as needed. The codecs in these systems seldom interact except by way of synchronizing, sharing resources, or transcoding, and are usually limited to audio codecs.
U.S. Pat. No. 7,457,359 recites systems and methods for securely distributing highly-compressed multimedia where, for each segment of video, a codec that best satisfies some set of criteria is selected from a pre-specified collection of codecs. Thus, each competing encoder operates individually on each segment, in parallel with its competitors. The encoding selected for a segment is the one that best endows that segment with the desired features, sacrificing features possessed by other encodings. In other words, feature trade-offs occur segment-by-segment instead of once, for the full video—but the features are still sacrificed. Furthermore, every segment in the video must be tested and compared using every codec in the codec set, adding to encoding time accordingly. Also, to decode the video, each viewer must have access to a decoder for each pre-specified codec. This method may also pose segment transition quality maintenance problems, synchronization issues, and other problems involving visually noticeable transitions between various codecs. Moreover, many features (e.g., scalability) must apply to the entire video to be of value. In short, this approach does not provide much practical relief from having to select a single video encoder and decoder to the exclusion of other codecs.
U.S. Pat. No. 7,657,651 recites a similar idea—multiple encodings that allow the system to stream the encoding that best serves the individual client among a clientele with heterogeneous needs. This patent postulates a network over which broadcaster streams live media to a heterogeneous clientele. The broadcaster is prepared with multiple alternative data streams. Based on varying clientele needs and preferences, each client receives one suitable data stream at any one time although the selection can vary from one data stream to another over time. As in the previous patent, there is no concept of merging multiple same-frame encodings into a single frame. If multiple encodings are involved, each is used by itself, with no direct interaction at any time with any other encoding. Moreover, all applications within the scope of this patent involve live broadcast, a plurality of users, and user feedback that determines which encoding from the predetermined set of alternate encodings will be received, none of which addresses the need for codec systems that combine, frame-by-frame, the best features of multiple codecs.
U.S. Pat. No. 7,920,633 recites a method for parallel processing that sets compression parameters for a group of pictures (GOP) to be processed by a third encoder based on those used to process a first GOP by a first encoder and those used to process a second GOP by a second encoder. This patent claims to reduce encoding time and encourages better compression-quality trade-offs both by making compression comparisons easier and by using the best choice of two or three encoders. However, this patent does not suggest non-parallel applications, non-compression/quality applications, multiple codecs with interacting subsystems, same-frame encodings by multiple codecs to be merged into a single frame, or other features of video multi-codec encoders.
To overcome the problems and limitations described above, there is a need for video multi-codec encoders, including video multi-codec encoders that synergize advantageous features of multiple codecs into a single system, the encoded video products of which have critical advantages over those of the simple existing encoder applied to the same video input data.